1. Field of Invention
The present invention relates to vision display systems used to provide a user with a visual display of a field of interest, and more particularly to systems and methods that combine images from a plurality of sources to provide a coherent view of a field of interest.
2. Description of Related Art
Despite the advent of many flight navigational aids, one of the most important tools for navigation of aircraft remains visual navigation. Many of today's aircrafts include various safety features such as on board radar, ground proximity warning systems, etc. that provide a pilot with added information about the airspace surrounding the aircraft. These systems are a tremendous resource to aid the pilot in obtaining a better situational awareness during flight, by allowing the pilot to further interpret what he or she is visually observing. However, there are instances where these various instruments become the pilot's only resource of information because the pilot's vision is hindered.
Visual hindrances may be due to bad weather, such as fog, snow, or rain, or they may be due to the time of day, such as night, dawn, or dusk. Further, some visual hindrances are due to the field of view limitations of the aircraft itself. Many aircraft cockpits have a field of view that is typically limited to a forward facing area that does not provide the pilot with adequate visualization to the sides and rear of the aircraft and also does not provide adequate vertical visualization above and below the aircraft.
Obstructed vision is an important safety concern in aircraft navigation, and there has been considerable effort devoted to providing systems that increase or enhance a pilot's view from the cockpit. Systems have been developed that include the use of one or more sensors that are located on the aircraft. The sensors are directed toward a selected field of view and provide images to a display system in the cockpit, where they are, in turn, displayed to the pilot. The sensors may be video cameras, infrared cameras, radar, etc. The systems allow the pilot to choose the types of images to view. For example, in nighttime flight or fog conditions, the pilot may opt to view images from the infrared and radar sensors, while under clear conditions, the pilot may use video camera feeds.
These systems may also include synthetic image sources. Specifically, many systems include mapping databases that include synthetic illustrations of various geographic features. These mapping databases can be coordinated with the actual position of the aircraft so that the synthetic images may be displayed to give the pilot a synthetic visualization of the terrain within the range of the aircraft.
One example of an enhanced vision system is disclosed in U.S. Pat. No. 5,317,394 to Hale et al., which is incorporated herein by reference. In this system, sensors are positioned on the exterior of the aircraft such that adjacent sensors have overlapped fields of view. Images from these various sensors are provided to a display system in the aircraft, where they are displayed to the pilot. The images are displayed in an overlapped configuration so as to provide a composite or mosaic image.
A more advanced system is disclosed in U.S. patent application Ser. No. 09/608,234, entitled: Exterior Aircraft Vision System Using a Helmet-Mounted Display, which is incorporated herein by reference. The Ser. No. 09/608,234 application discloses a system that includes a helmet-mounted display for displaying images from various sensors located on the aircraft. Importantly, this system includes a helmet-tracking device that tracks movement of the pilot's head in order to determine the pilot's current line of sight (LOS) and field of view (FOV). Using this directional information, the system retrieves image data from the sensors that represent the field of view in which the pilot is staring and displays this image on the helmet display. The image is updated as the pilot turns his head to different lines of sight.
In general, these and other conventional systems provide fairly accurate visual images to the pilot, and thereby increase flight safety. However, there are some limitations to these systems that can cause the images provided to the pilot to either be less accurate or include anomalies that may distract the pilot's view. For example, one issue relates to the spacing of sensors relative to each other and relative to the pilot's position or in case of stored synthetic data, the difference in perspective between the synthetic data and the other sensors and pilot position. This physical distance between the sources relative to each other and to the cockpit may cause a skewing of the images provided to the pilot. Specifically, the distance creates a visual skewing in the images referred to as parallax. Parallax is an apparent change in the direction of an object caused by a change in observational position that provides a new line of sight. In these conventional systems, the sensor or sensors and/or synthetic data source each have a different line of sight with regard to a scene from that of the pilot. As such, when viewing the images from the sensor point of view or point of view of the synthetic data, the pilot is not provided with an accurate representation of the location of the object relative to his position in the aircraft in their overlapping fields of view. Also, the parallax effect between sensors and/or synthetic data can cause ghosting and other phenomena.
Another issue relates to tiling of several images together to create a composite image. In many conventional systems, images from adjacent cameras are displayed adjacent to each other. The edges of the two images appear as visual seams in the display. These seams disrupt viewing of the composite image and can make it harder for the pilot to view the image as a whole. To correct this problem some prior art systems overlap the edges of adjacent images in an attempt to blend the images together. While this technique is an improvement over the conventional technique of abutting adjacent images, there may still be perceived discontinuity between the images.
A further issue relates to the limitations of a particular type of sensor to provide the best imaging for a given situation. For example, in twilight conditions, a video camera will still provide a discernable visual image, but the image will be degraded in detail due to the low light or obstructed conditions. Further, an infrared sensor will provide imaging based on heat sensing, but the image from an infrared sensor will not have the benefit of the ambient light still available at twilight. In many conventional systems, the pilot must select between these images, instead having an image available that incorporates the advantages of both sensors. As such, the pilot does not have the best images available for viewing.